Code Swaraj by Sam Pitroda & Carl Malamud & Carl Malamud & Sam Pitroda

Author:Sam Pitroda & Carl Malamud & Carl Malamud & Sam Pitroda [swaraj.epub]
Language: eng
Format: epub
Publisher: Public.Resource.Org
Published: 2018-02-05T00:00:00+00:00

Back To Work On Real Data, My “Bread Labor”

As November came to a close, I continued to work on my backlog from India. Most pressing was the Digital Library of India, which I had renamed the Public Library of India. The government still hadn’t put their version back online and Sanskrit scholars were sending me notes asking for additional materials. With the addition of 4,450 books from the Archeological Survey of India, we had pushed the total volumes online close to 400,000 books.

Also taking considerable time were the official gazettes of India. The national government gazette had been straightforward to mirror. Looking in the Public Library repository, I found several hundred old gazettes from before liberation, and those were also added to the collection. The tough part though, were the gazettes of the state governments and several large cities.

One example was the Odisha Gazette, the official publication of a state with 43 million people. I wrote a script that brought down 38,073 issues of the gazette as PDF files. But, after that script ran, I pulled up some of the files and noticed that they referenced a font for the language Oriya that was not embedded in the PDF file. That means all you see is gibberish because your computer is looking for that font installed on the system instead of embedded in the file.

After running a series of scripts, I determined that 35,705 of the files had this problem and I would have to embed the fonts before uploading them to the Internet Archive. But, the font that they assumed would be on your system is an obscure one that was produced years ago by an Indian research institute and after several days of looking around, I was unable to find it for sale or for download, so I shelved Odisha for the time being.

Other states were even harder. With Odisha, I was able to pull up index files with long lists of issues of the Gazette, and in that index file was a URL directly to each PDF file. By first bringing down the indices, then parsing them for metadata and file addresses, it was fairly trivial to bring in all the PDF files. But, most of the states were not that straightforward.

Most of the state gazettes are based on some Microsoft server software which does not expose the URL (network address) of the PDF files. The problem was that each state had a different opaque way of getting issues. There are several dozen official gazettes in India, one for each state and ones for major municipalities like Delhi. Each one is programmed differently.

We had amassed 163,977 total PDF files in the collection, but it was clear that to do this right, we would have to do some serious work on it in 2018. Not only did files have to be brought in for all the gazettes, the collection needed to be kept up to date to be truly useful, and in order to permit the kind of

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.